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(54) [Title of the Invention] 

INFORMATION EMBEDDING METHOD AND APPARATUS 
[Abstract] 

[Problem] To providie a method for embedding watermark 
information, with suppressed deterioration of contents 
quality and increased robustness of embedded information 
[Solving problem] When watermark information is embedded 
in moving image data consisting of plural static image 
frames arranged in time series, motion vectors (moving 
image properties) are detected for each image block in a 
noticed frame, a specification rule 320 of an image 
change rate of each block is selected according to a 
motion amount, as many pixels as specified in the rule 
are selected from brightness changeable pixels dependent 
on image state (static image properties) within each 
block, and brightness changes as watermark information 
are made . 
[Claims] 

[Claim 1] An information embedding method comprising the 
steps of: 

for plural properties of contents information in 
which watermark information is to be embedded, obtaining 
change locations and change degrees dependent on each 
property; and 

from plural sets of change locations and change 
degrees obtained depending on the each property, 
selecting one set of change locations and change degrees 
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to be applied to the contents information, 

wherein changes as watermark information are made 
to the contents information according to the selected 
change locations and change degrees. 

[Claim 2] The information embedding method according to 
claim 1, wherein the contents information in which the 
watermark information is to be embedded is split into 
plural blocks, the plural sets of change locations and 
change degrees are obtained for each information block, 
and one set of change locations and change degrees are 
selected for the each information block. 

[Claim 3] An information embedding method comprising: 

a first step of obtaining change locations and 
change degrees, depending on a first property of 
contents information in which watermark information is 
to be embedded; and 

a second step of modifying at least part of the 
change locations and change degrees obtained in the 
first step, depending on a second property different 
from the first property of the contents information, 

wherein changes as watermark information are made 
to the contents information according to the modified 
change locations and change degrees. 

[Claim 4] An information embedding method comprising: 

a first step of selecting one of plural rules 
provided in advance, depending on a first property of 
contents information in which watermark information is 
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to be embedded; 

a second step of obtaining change location 
candidates of the contents information, depending on a 
second property of the contents information; and 

a third step of selecting at least one change 
location from the change location candidates, based on 
the rule selected in the first step, 

wherein the state of the contents information is 
changed in the selected change locations. 

[Claim 5] The information embedding method according to 
claim 4, wherein the contents information in which the 
watermark information is to be embedded is split into 
plural blocks, one of the plural rules is selected for 
each information block, the change location candidates 
are obtained for the each information block, and change 
locations are selected according to the selected rule. 
[Claim 6] The information embedding method according to 
claim 4 or 5, wherein: the plural rules define a 
relationship between a first parameter indicating the 
ease of change to the contents information, and a change 
rate; in the third step, the value of the first 
parameter indicating the ease of change is calculated 
based on the change location candidates obtained in the 
second step; and from the change location candidates, as 
many change locations as indicated by a change rate 
corresponding to the first parameter value indicated by 
the selected rule are selected. 
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[Claim 7] An information embedding method that embeds 
watermark information by making changes to part of 
moving image data consisting of plural static image 
frames, the method comprising: 

a first step of obtaining change locations and 
change degrees, depending on static image properties 
detected from a static image frame to be processed; 

a second step of obtaining change locations and 
change degrees, depending on moving image properties 
detected from a relationship between images contained in 
the static image frame to be processed and another 
static image frame; and 

a third step of selecting sets of change locations 
and change degrees to be applied to the static image 
frame from the change locations and change degrees 
obtained in the second step, 

wherein pixel changes as watermark information are 
made to the static image frame to be processed according 
to the selected change locations and change degrees. 
[Claim 8] The information embedding method according to 
claim 7, wherein the each static image frame is split 
into plural image blocks, and the first, second, and 
third steps are performed for each image block. 
[Claim 9] An information embedding method that embeds 
watermark information by making changes to part of 
moving image data consisting of plural static image 
frames, the method comprising: 
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a first step of obtaining change locations and 
change degrees, depending on static image properties of 
the each static image frame; and 

a second step of modifying at least part of change 
locations and change degrees obtained in the first step, 
depending on moving image properties obtained from a 
relationship between the static image frames, 

wherein pixel changes as watermark information are 
made to the static image frames according to the 
modified change locations and change degrees. 
[Claim 10] An information embedding method that embeds 
watermark information by making changes to part of 
moving image data consisting of plural static image 
frames, the method comprising: 

a first step of selecting one of plural rules 
provided in advance, depending on moving image 
properties detected from a relationship between images 
contained in a static image frame to be processed and 
another static image frame; 

a second step of obtaining change location 
candidates in the static image frame, depending on 
static image properties detected from the static image 
frame to be processed; and 

a third step of selecting at least one change 
location from the change location candidates, based on 
the rule selected in the first step, 

wherein pixel states of the static image frame are 



JP-A NO. 2000-175019 



changed in the selected change locations. 

[Claim 11] The information embedding method according to 
claim 10, wherein: the each static image frame is split 
into plural image blocks; one of the plural rules is 
selected for each image block; the change location 
candidates are obtained for the each image block; and 
change locations are selected according to the selected 
rule . 

[Claim 12] An information embedding apparatus, 
comprising: 

means, for plural properties of contents 
information in which watermark information is to be 
embedded, for obtaining change locations and change 
degrees dependent on each property; 

means, from plural sets of change locations and 
change degrees obtained depending on the each property, 
for selecting one set of change locations and change 
degrees to be applied to the contents information; and 

means for making changes as watermark information 
to the contents information according to the selected 
change locations and change degrees . 
[Detailed Description of the Invention] 
[0001] 

[Technical Field of the Invention] The present invention 
relates to an information embedding method and apparatus, 
and more particularly to a method for embedding copy 
control information, copyright information, and other 
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information in contents information such as digitized 
static images or moving images, and an apparatus for the 
same . 
[0002] 

[Prior Art] In recent years, digitized contents such as 
images and music have been delivered through recording 
media or communication networks. To protect the 
copyright of such digitized contents information, an 
electronic watermark technique is receiving attention. 
According to this technique, such minor changes that 
will not be perceptible to human eyes are made to 
contents information to embed copy control information 
and copyright information, thereby making it possible to 
limit the number of times the contents information is 
copied using data processing apparatuses, and identify a 
copyright holder from contents information illegally 
copied. 

[0003] Actual use of the watermark technology 
requires that the following two requirements be 
satisfied. 

(1) Suppression of deterioration of contents 
information: For example, electronic watermark 
information intended for image data must be embedded so 
that changes made to the images do not interfere with 
contents appreciation. In other words, changes made to 
part of contents information must be audiovisually 
inconspicuous . 
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[0004] (2) Increase in robustness: In spite of image 
processing and sound processing, embedded information, 
that is, changes made to contents information must be 
almost free from deterioration. To satisfy the above- 
described conditions, according to the properties of 
contents, contents change locations, and change degrees 
or intensity must be optimized. Hereinafter, 
conventional electronic watermark techniques are 
analyzed using an example of images, which are 
representative as contents information. 

[0005] Image data generally has the following property. 
In areas where image states change relatively smoothly, 
if inappropriate changes were made to the state values 
(brightness or color) of pixels, the change locations 
would be unnaturally conspicuous. In contrast, in 
subject edges and other portions where pixel values 
changes heavily, even if relatively major changes were 
made to the pixels, the change locations would be 
inconspicuous to human eyes. An edge preservation 
electronic watermark technique based on this property is 
proposed in Information Processing Institute Paper, Vol. 
38, No. 12, 1997, pages 2640 to 2647. According to this 
technique, subject edges are located by analyzing 
brightness variations of image frames to be embedded 
with watermark information, major changes are made in 
the edges where the changes to pixel values will be 
inconspicuous, and no changes but minor changes if made 
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are made in portions where the changes to pixel vaiues 
will be conspicuous . 

[0006] Electronic watermark information is embedded not 
only in static images but also in moving images. Moving 
image data consists of plural static image frames 
arranged in time series. By stopping advancing frames 
during appreciation of moving images, individual frames 
may be appreciated as static images. 

[0007] 

[Problem to be Solved by the Invention] For the above- 
described reason, in electronic watermark information 
embedding intended for moving images, it is necessary to 
make change locations of pixel values inconspicuous in 
both static images and moving images, and it is 
desirable to optimize change pixels and degrees in 
consideration of both static image properties and moving 
image properties. However, conventional techniques for 
electronic watermark information embedding intended for 
moving images take only static image properties or 
moving image properties into account; for example, the 
method reported in 19 95 Symposium on Cryptography and 
Information Security, 31-G takes only static image 
properties into account, and the method reported in 1995 
Symposium on Cryptography and Information Security, 31-F 
takes only moving image properties into account. There 
has not yet been reported an example that optimizes 
change locations and change intensity in consideration 
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of both static image properties and moving image 
properties . 

[0008] The consciousness of pixel value change depend on 
not only one property such as brightness variations but 
also properties such as color variations other than 
brightness. Therefore, when watermark information is 
embedded in static images or moving images, the plural 
properties must be taken into account to optimize change 
locations and change degrees (change intensity) . However, 
conventional electronic watermark techniques rely on one 
property of images for optimization of pixel value 
change; no optimizing technique taking plural properties 
into account has been reported. 

[0009] A first object of the present invention is to 
optimize change locations and change intensity in 
consideration of plural properties of contents 
information in which watermark information is to be 
embedded. A second object of the present invention is to 
provide a method for embedding watermark information in 
moving images by change locations and change intensity 
optimized in consideration of both moving image 
properties and static image properties, and an apparatus 
for the same. 

[0010] 

[Means for Solving the Problem] To achieve the first 
object, an information embedding method of the present 
invention comprises the steps of: for plural properties 
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of contents information in which watermark information 
is to be embedded, obtaining change locations and change 
degrees dependent on each property; and from plural sets 
of change locations and change degrees obtained 
depending on the each property, selecting one set of 
change locations and change degrees to be applied to the 
contents information, wherein changes as watermark 
information are made to the contents information 
according to the selected change locations and change 
degrees. In actual application, the contents information 
in which the watermark information is to be embedded is 
split into plural blocks, the plural sets of change 
locations and change degrees are obtained for each 
information block, and one set of change locations and 
change degrees are selected for the each information 
block. 

[0011] In the selection step, if higher priority is 
placed on the inconspicuousness of changes, the results 
of AND operations with respect to plural sets of change 
locations selected, for example, for each property may 
be obtained as final change locations so that the number 
of change locations is minimum. Conversely, if higher 
priority is placed on the robustness of change results, 
the results of OR operations with respect to plural sets 
of change locations selected for each property may be 
obtained as final change locations so that the number of 
change locations is maximum. 
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[0012] 

To achieve the first object, the information 
embedding method according to another embodiment of the 
present invention includes: a first step of obtaining 
change locations and change degrees, depending on a 
first property of contents information in which 
watermark information is to be embedded; and a second 
step of modifying at least part of the change locations 
and change degrees obtained in the first step, depending 
on a second property different from the first property 
of the contents information, wherein changes as 
watermark information are made to the contents 
information according to the modified change locations 
and change degrees . 

[0013] To achieve the first object, the information 
embedding method according to still another embodiment 
of the present invention includes: a first step of 
selecting one of plural rules provided in advance, 
depending on a first property of contents information in 
which watermark information is to be embedded; a second 
step of obtaining change location candidates of the 
contents information, depending on a second property of 
the contents information; and a third step of selecting 
at least one change location from the change location 
candidates, based on the rule selected in the first step, 
wherein the state of the contents information is changed 
in the selected change locations. According to the 
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present invention, each of the plural rules defines 
change locations and change degrees according to one 
property of the contents information, and by applying 
one rule selected depending on a different property of 
the contents information, change locations and change 
degrees can be decided in consideration of plural 
properties of the content information. 

[0014] The second object is achieved by an information 
embedding method of the present invention that embeds 
watermark information by making changes to part of 
moving image data consisting of plural static image 
frames, the method comprising: a first step of obtaining 
change locations and change degrees, depending on static 
image properties detected from a static image frame to 
be processed; a second step of obtaining change 
locations and change degrees, depending on moving image 
properties detected from a relationship between images 
contained in the static image frame to be processed and 
another static image frame; and a third step of 
selecting sets of change locations and change degrees to 
be applied to the static image frame from the change 
locations and change degrees obtained in the second step, 
wherein pixel changes as watermark information are made 
to the static image frame to be processed according to 
the selected change locations and change degrees. 
[0015] The second object is achieved by an information 
embedding method according to another embodiment of the 
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present invention that embeds watermark information by 
making changes to part of moving image data consisting 
of plural static image frames, the method comprising: a 
first step of obtaining change locations and change 
degrees, depending on static image properties of the 
each static image frame; and a second step of modifying 
at least part of change locations and change degrees 
obtained in the first step, depending on moving image 
properties obtained from a relationship between the 
static image frames, wherein pixel changes as watermark 
information are made to the static image frames 
according to the modified change locations and change 
degrees . 

[0016] The second object is achieved by an information 
embedding method according to still another embodiment 
of the present invention that embeds watermark 
information by making changes to part of moving image 
data consisting of plural static image frames, wherein 
the method comprising: a first step of selecting one of 
plural rules provided in advance, depending on moving 
image properties detected from a relationship between 
images contained in a static image frame to be processed 
and another static image frame; a second step of 
obtaining change location candidates in the static image 
frame, depending on static image properties detected 
from the static image frame to be processed; and a third 
step of selecting at least one change location from the 
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change location candidates, based on the rule selected 
in the first step, wherein pixel states of the static 
image frame are changed in the selected change locations 
[0017] In actual application, the each static image 
frame is split into plural image blocks and the above- 
described series of processing steps are performed for 
each image block. As static image properties, for 
example, variations in brightness within each static 
image frame or each image block may be considered. As 
moving image properties, an image motion amount, that is 
the magnitude of motion vectors between static image 
frames may be considered. 

[0018] An information embedding apparatus of the present 
invention comprises: a processor for executing the 
above-described steps by program processing; and a 
storage device for storing contents information to be 
processed, and other data. Its concrete configuration 
will become more apparent from embodiments described 
below. 

[0019] 

[Detailed Description of the Invention] A first 
embodiment of the present invention will be described 
with reference to FIGS. 1 to 8 . In the present 
embodiment, moving images are taken up as contents 
information. Moving image data consists of plural time- 
series static image frames. By making changes to part of 
pixels of each static image frame, watermark information 
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can be embedded in the moving image. In embodiments 
described below, for specific pixels selected from each 
static image frame, their brightness is changed in a 
range that does not give human eyes a sense of 
incongruity. 

[0020] FIG. 1 shows a hardware configuration for 
embedding watermark information according to the present 
invention. An input -output device 1 inputs moving image 
data to embed watermark information in, watermark 
information, and data of various rules and the like 
described later, and outputs a moving image in which the 
watermark information is embedded. 
Numeral 2 designates a central processing unit 
(processor) . A storage device 3 stores moving image data 
inputted from the input -output device and moving image 
data in which watermark information is embedded. The 
storage device 3 also stores various types of data, 
rules, programs, and the like used by the central 
processing unit, in addition to moving image data 
required to embed watermark information. In actual use, 
a storage device for storing programs and computer data 
is provided aside from a large-capacity storage device 
for storing images. However, in this example, one 
storage device is shown for the sake of simplicity. 
[0021] The central processing unit detects subject's 
motion (moving image property) by 

comparing individual static image frames constituting a 
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moving image and following frames appearing 
predetermined frames behind on a time axis and, in 
consideration of both the property of static image of 
each frame and the moving image property obtained in 
motion detection, selects pixels in which watermark 
information is to be embedded, and decides change 
amounts of pixel values . 

[0022] FIG. 2 is a functional block diagram showing a 
first embodiment of electronic watermark information 
embedding of the present invention implemented through 
program operation performed by the central processing 
unit 2. Functional blocks 20 to 25 designate processing 
routines (programs) executed by the central processing 
unit 2, and functional blocks 31 to 3 5 designate data 
files formed within the storage device 3. For example, 
31 designates a moving image file in which moving image 
data consisting of plural static image frames is stored; 
32, a rule aggregation file for storing plural rules 
shown in FIG. 5; 33, a use rule file for storing rules 
to be applied for each of image blocks of predetermined 
size obtained by splitting the each static image frame; 
34, an embedding information file for storing watermark 
information to be embedded in static images constituting 
a moving image; and 35, an information embedding moving 
image file for storing moving image data in which 
watermark information is embedded. 

[0023] An input-output routine 21 writes moving image 
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data, rule data, and watermark information inputted from 
the input-output device 1 by a user to the files 31, 32, 
and 34, respectively; in response to a request from the 
user, reads moving image data embedded with watermark 
information from the information embedding image file 
35; and outputs it to a display screen of the input- 
output device 1. A control routine 20, in conjunction 
with the input-output routine, for each of static image 
frames constituting a moving image, successively starts 
an inter-static-image motion detection routine 22, a 
rule selection routine 23, a change location decision 
routine 24, and a pixel state change routine 25, and 
embeds watermark information. 

[0024] The inter- static - image motion detection routine 
22 compares a static image (hereinafter referred to as a 
noticed frame) noticed as a target for embedding 
watermark information and a static image (hereinafter 
referred to as a reference frame) appearing K frames 
behind on a time axis, and detects the motion of a 
subject contained in the noticed frame. The motion 
detection processing, as detailed in FIG. 3, splits a 
noticed frame into plural image blocks of predetermined 
size, for example, 16 pixels by 8 pixels (= 128 pixels) 
each, and finds a motion vector with a reference block 
for each image block. The value (e.g., K=10) of a 
parameter K indicating a frame interval between the 
notice frame and the reference frame can be freely 
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specified by users of the system. 

[0025] The rule selection routine 23 receives a motion 
vector of each image block from the inter- static- image 
motion detection routine 22, selects one of plural rules 
stored in the rule aggregation file 32 according to the 
size of the motion vector, and stores it in the use rule 
file 33 as a block- specif ic application rule. 
[0026] The change location decision routine 24, as 
detailed in FIG. 7, analyzes a static image contained in 
a noticed frame, finds a change ease level for each 
image block, and decides brightness change locations 
(pixels) and a change amount of each image block within 
the noticed frame according to the change ease level and 
an application rule taken out from the use rule file 33. 
The change routine 2 5 receives the brightness change 
locations (pixels) and change amount of each image block 
within the noticed frame from the change location 
decision routine 24, and changes the state (brightness) 
of pixels within the image block according to watermark 
information specified in the embedding information file 
34 . 

[0027] For example, when each of the static images 
constituting the above-described moving image consists 
of 720 by 480 pixels, if it is split in units of by 16 
by 8 pixels, one frame is split into blocks of 45 by 60 

(= 2700) . If watermark information embedded in the frame 
is 6 -bit information (64 types of character code 
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represented by values 0 to 63) , 450 blocks can be 
assigned to each bit. If 2700 blocks constituting one 
frame are divided into N groups corresponding to N 
character codes and 2700/N blocks are assigned to the 
above-described 6-bit information, 450/N blocks can be 
assigned per bit. 

[0028] Individual bits of a bit pattern constituting the 
watermark information are associated in advance with 
plural image blocks in mutually discrete positions 
within a static image frame. For example, if bit 
information "1" is written to a certain bit position, in 
plural image blocks corresponding to the bit position, 
brightness is increased by a specified amount in pixel 
positions specified by the change location decision 
routine 24, and if bit information "0" is written, 
brightness is decreased by a specified amount in pixel 
positions specified by the decision routine 24. Thereby, 
N character codes can be embedded in each frame as 
watermark information. Although blocks whose brightness 
cannot be changed may exist depending on the state of an 
image within a frame, if plural image blocks in mutually 
discrete positions within the frame are assigned to each 
bit, brightness can probably be changed in at least one 
block, ensuring that watermark information can be 
embedded . 

[0029] The correspondence between the above-described 
image blocks and bit information is stored in the 
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embedding information file 35 in the form of table when 
the watermark information is afforded, in order that the 
change routine 2 5 can determine write information of 
each block by consulting the embedding information file 
35. 

[003 0] As the watermark information, 8 -bit information 
may be used instead of the 6-bit information. Instead of 
associating each bit and image blocks as described above, 
for example, 64 types of character codes representable 
by the 6 -bit information and image blocks may be 
associated so that the pixel state of corresponding 
image blocks is changed depending on the existence or 
absence of character codes. In the case where the 
brightness of each pixel has 128 levels from 0 to 127, 
instead of changing the brightness level of each pixel 1 
specified as a change location by a specified amount, 
the brightness change of each pixel may be fixed to one 
level so that the number of pixels whose brightness is 
to be changed is variable according to the specified 
amount . 

[0031] FIG. 3 shows a detailed flowchart of the inter- 
static-image motion detection routine 22. Motion 
detection required can be realized by known technique 
disclosed by "Basic of Digital Image Compression", pp. 
44-47, 1996 published by Nikkei BP publication Center. 
The result of motion detection is represented by a least 
square sum of differences between motion vector and 
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subject image. By using the values of the both, accurate 
information embedding control can be conducted according 
to the motion of images. However, here, for simplicity 
of explanation, motion detection results are represented 
by motion vectors instead of a least square sum. 
[0032] As shown in the drawing, a back frame positioned 
K frames behind a noticed frame is fetched from the 
moving image file 31 (step 221) . The noticed frame is 
split into blocks of 16 pixels by 8 pixels each (step 
222) . The first block of the noticed frame is noticed 
(step 223) . Motion between the noticed block and the 
back frame is detected, and an obtained motion vector is 
passed to the rule selection routine 23 (step 224) . it 
is judged whether all blocks of the noticed frame have 
been processed (step 225) . if all blocks have been 
processed, this routine is terminated; otherwise, the 
next block is selected as a noticed block (step 226). 
The motion vector detection step 224 is repeated. 
[0033] As a result of the motion detection between 
static images, as shown in FIG. 4, a vector table 220 
showing the correspondence between blocks 22 0a of a 
noticed frame and motion vectors 220b is formed, and its 
contents are afforded to the rule selection routine 23 
from the detection routine 22. 

[0034] The rule selection routine 23, for example, as 
shown in FIG. 5, has plural judgment criteria Rl, R2, 
R3,... and forth for determining a watermark intensity 
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specification table to be selected according to the 
magnitude of motion vector, fetches a rule corresponding 
to a motion vector amount (absolute value) detected in 
the motion detection routine 22 for each block within a 
noticed frame from the rule aggregation file 32, and 
stores it in the rule selection routine 23 in 
association with an image block. 

[0035] Rules (intensity specification table) 320 
provided in the rule aggregation file 32 define 
relationships between change ease levels 320a of blocks 
in static images and block change rates 320b, as shown 
in FIG. 6. The change ease levels 320a indicate the 
level of brightness change that would be inconspicuous 
for each blocks; in this embodiment, they indicate an 
average value per pixel of a brightness change amount 
permitted for each block, that is, at which of the 128 
levels a brightness change is permitted per pixel. The 
block change rates 320b indicate what percent of pixels 
within one block is changed. As described later, when 
plural pixel candidates whose state can be changed are 
found in one block of a static image frame, as many 
pixels as specified in the above-described change rates 
are selected as pixels eligible for brightness change 
from the plural pixel candidates. Therefore, the above- 
described change rates designate the positions of change 
pixels and a change level of pixels within a block at 
the same time. 
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[0036] The judgment criteria Rl, R2, R3 , . . . and so forth 
represent relationships between motion vectors and rules 

(intensity specification table 320) so that blocks 
having large motion vectors are subjected to application 
of rules having higher block change rates for an 
identical change ease level. 

[0037] FIG. 7 shows a detailed flowchart of the change 
location decision routine 24. For each block of a 
noticed frame, a change ease level is determined 
according to the following procedure, for example (step 
241) . For each noticed frame, image data (hereinafter 
referred to as a difference image frame) indicating a 
change amount is created by performing noise elimination 
processing (pixel value change) by image filtering 
having properties described in Electronic Information 
Communication Institute Paper, D-2, Volume J79-D-2, 
Eighth Number, pages 1347 to 1353, within a range that 
is not perceptible to human eyes and does not interfere 
with contents reference, and taking a difference between 
the filtered image and an original image. Since the 
above-described filtering is for image processing in a 
range that does not interfere with contents reference, 
the difference image frame indicates candidate positions 
and change amounts of pixel value change permitted for a 
noticed frame. Accordingly, the difference image frame 
is split into plural blocks like a noticed frame, a 
cumulative value of pixel values (brightness change 
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amounts) is found for each block, and it is divided by 
the number of block pixels to find an average brightness 
change amount per pixel of each block, that is, a change 
ease level 320a shown in FIG. 6. 

[0038] Next, in each block of the difference image frame, 
by selecting pixels in descending order of pixel value, 
the change ease rank of pixel is determined (step 242) . 
When changes are made to two pixels pi and p2 within a 
block with an identical brightness amount, if pi is 
higher than p2 in change ease rank, pi is more 
inconspicuous than p2 in brightness change. 

[0039] In this state, the first block of the noticed 
frame is noticed (step 243) . A rule (table 320) 
corresponding to the noticed block is read from the use 
rule file 33, and according to the rule, a block change 
rate 320b corresponding to the change ease level 320a of 
the first block, already calculated in the step 241, is 
obtained (step 244) . This determines how many pixels of 
128 pixels within the noticed block are to be changed. 
Next, from the 128 pixels of the noticed block, as many 
pixels as indicated in the block change rate 320b are 
selected in descending order of change ease rank 
determined in step 242, and the selected pixels are 
passed to the change routine 25 (step 245) . Whether all 
blocks of the noticed frame have been processed is 
determined (step 246) , and if all blocks have been 
processed, the routine is terminated; otherwise, a next 
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block within the noticed frame is selected as a noticed 
block (step 247), and control is returned to the change 
rate decision step 244. 

[0040] The change routine 25 consults the embedding 
information file 34 to locate watermark information to 
be written to the noticed block, and makes brightness 
changes corresponding to the watermark information for 
pixels indicated by the change location decision routine 
24, of the 128 pixels of the noticed block. In the 
above-described embodiment, a change amount of pixel 
brightness is controlled by fixing it to one level and 
changing the number of brightness change pixels within 
the noticed block. However, as described previously, a 
brightness change amount of each pixel may be variably 
controlled. 

[0041] As described above, in this embodiment, rules to 
be applied are selected depending on motion vectors 
(moving image property) indicating relationships between 
static image frames constituting moving images, and 
pixels whose brightness is to be changed are decided 
based on the application rules from pixels whose 
brightness can be changed, determined depending on the 
state (static image property) of subject contained in 
each block within a static image frame. Therefore, 
according to this embodiment, pixel state change 
positions and change amounts for embedding watermark 
information are decided in consideration of both the 



JP-A NO. 2000-175019 



properties of moving images and static images so that 
watermark information can be embedded with suppressed 
deterioration of image quality and increased information 
robustness . 

[0042] FIG. 8 shows a variant of the rules 320 shown in 
FIG. 6. In this variant, correspondingly to block change 
ease levels 320a, brightness change amounts 320c for 
each pixel are defined in addition to block change rates 
320b, whereby, in moving frames where brightness change 
will be inconspicuous, watermark information can be 
embedded with an increased number of change pixels and 
increased change amounts of individual pixels. 
[0043] FIG. 9 is a functional block diagram showing a 
second embodiment of electronic watermark information 
embedding of the present invention. Functional blocks 20 
to 28 designate processing routines (programs) executed 
by the central processing unit 2, and functional blocks 
31 to 37 designate data files formed within the storage 
device 3. Components common to those in the first 
embodiment described in FIG. 2 are identified by the 
same reference numerals. In the first embodiment, plural 
rules are provided in the rule aggregation file 32, and 
for each of blocks of a noticed frame, application rules 
are selected according to motion vectors. However, in 
this embodiment, one rule 320 stored in advance in the 
use rule file 33 is applied to all blocks. 
[0044] A change rate 1 decision routine 26 decides the 
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block change rate of each noticed block, depending on 
the magnitude of motion vector of each block within a 
noticed frame received from the inter-static- image 
motion detection routine 22, and stores the result in a 
change rate 1 file 3 6A. The block change rate can be 
decided by consulting a conversion table defining in 
advance a relationship between the magnitude of motion 
vectors and change rates so that change rates become 
higher as motion vectors become larger. 
[0045] A static image analysis routine 27 calculates 
change ease level 320a of each block by the same 
processing that the change location and amount decision 
routine 24 described in FIG. 2 performs, decides change 
rate 320b within a noticed block according to the rule 
320 provided in the use rule file 33, and stores the 
result in a change rate 2 file 36B. 

[0045] In this embodiment, in this way, for an identical 
noticed block, one of change rate 1 decided according to 
motion vectors having moving image properties, and 
change rate 2 decided according to change ease levels 
having static image properties is selected by a change 
rate decision routine 28. The change rate decision 
routine 28 selects the smaller of the above-described 
two change rates, for example, to give higher priority 
to image quality after brightness change, and stores it 
in a change rate file 37. If higher priority is to be 
given to the robustness of watermark information after 
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embedding, the greater of the above-described two change 
rates is selected. 

[0047] A change location decision routine 24' finds 
pixels within a noticed block whose brightness can be 
changed, by the same method as the change location 
decision routine 24 in the first embodiment, selects a 
predetermined number of pixels in descending order of 
change ease rank, using change rates set in the change 
rate file 37 instead of application rules of the use 
rule file 33, and passes them to the change routine 35 
as pixels subject to brightness change. 

[0048] As described above, one of a first change rate 
decided by motion vectors having moving image properties 
and a second change rate decided by image states having 
static image properties is selectively applied so that 
watermark information can be embedded with suppressed 
deterioration of image quality and increased information 
robustness. 

[0049] FIG. 10 is a functional block diagram showing a 
third embodiment of electronic watermark information 
embedding of the present invention. Functional blocks 2 0 
to 29 designate processing routines (programs) executed 
by the central processing unit 2, and functional blocks 
31 to 37 designate data file's formed within the storage 
device 3. Components common to those in the first and 
second embodiments described in FIGS. 2 and 9 are 
identified by the same reference numerals. 
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[0050] The static image analysis routine 27, as in the 
second embodiment, decides the change rates of each 
block, based on static image properties of a noticed 
frame, and stores it in a change rate 2 file 36B. A 
change rate adjustment routine 29 increases or decreases 
the value of change rate 2 read from the change rate 2 
file 3 6B, depending on motion vector of each block 
received from the inter-static-image motion detection 
routine 22, and stores it a change rate file 37 as an 
application change rate. A change location decision 
routine 24' decides pixels subject to brightness change 
as in the second embodiment, using change rates stored 
in the change rate file 37, and passes them to the 
change routine 25. 

[0051] In the third embodiment, by adjusting change 
rates 2 decided by static image properties according to 
motion vectors having moving image properties, final 
application change rates are found to decide brightness 
change pixels. 

[0052] In the first to third embodiments, a description 
was made of methods intended for moving images for 
embedding information deciding the embedding positions 
of watermark information in consideration of both moving 
image properties and static image properties. However, 
technical concept of the present invention is not 
limited to the above-described embodiments; for example, 
information embedding positions may be decided in 
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consideration of two different properties of contents 
such as brightness variation and color variation in 
watermark information embedding for static images, or 
the present invention may apply to other contents other 
than images, such as sound. Although, in the above - 
described embodiments, methods of the present invention 
are implemented by software (routines 20 to 29) 
execution by the central processing unit 2, part of 
functions of these software modules may be implemented 
by hardware . 
[0053] 

[Effect of the Invention] As apparent from the above 
description, according to the present invention, in 
consideration of plural properties of contents and 
optimizing state change locations and/or change levels, 
watermark information can be embedded with suppressed 
deterioration of contents values (image quality, sound 
quality, etc.) and increased robustness of embedded 
information. If the present invention is applied to 
moving image data consisting of plural static image 
frames arranged in time series, since change locations 
and change amount of pixel states can be optimized, 
robust watermark information free of apparent 
deterioration of image quality can be embedded. 

[Brief Description of the Drawings] 

[FIG. 1] Hardware configuration diagram for electronic 
watermark information embedding of the present invention 
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[FIG . 2] Functional block diagram showing a first 
embodiment of electronic watermark information embedding 
of the present invention 

[FIG . 3] Detailed flowchart of the inter-static-image 
motion detection routine 22 in FIG. 2 

[FIG. 4] Drawing showing results of motion detection for 
each block by the inter-static-image motion detection 
routine 22 

[FIG. 5] Drawing showing judgment criteria for rule 
selection that the rule selection routine 23 in FIG. 2 
has 

[FIG. 6] Drawing showing one example of information 
embedding rules 

[FIG. 7] Detailed flowchart of the change location 
decision routine 24 in FIG. 2 

[FIG. 8] Drawing showing another example of information 
embedding rules 

[FIG. 9] Functional block diagram showing a second 
embodiment of electronic watermark information embedding 
of the present invention 

[FIG. 10] Functional block diagram showing a third 
embodiment of electronic watermark information embedding 
of the present invention 
[Description of the Reference Numerals] 

1: input-output device, 2: Central processing unit, 3: 
Storage device, 20: Control routine, 21: Input-output 
routine, 22: Inter-static-image motion detection routine. 
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23: Rule selection routine, 24: Change location decision 
routine, 25: Change routine, 26: Change rate 1 decision 
routine, 27: Static image analysis routine, 28: Change 
rate decision routine, 29: Change rate adjustment 
routine, 31: Moving image file, 32: Rule aggregation 
routine, 33: Use rule file, 34: Embedding information 
file, 35: Information embedding moving image file, 36: 
Change rate file, 37: Application change rate file, 220: 
Motion vector storage table, 320: Change rate decision 
rule 
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FIG. 1 

1. Input-output device 

2. Central processing unit 
3 . Storage apparatus 
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FIG. 2 

21. Input-output 
20. Control 

22. Inter-static-image motion detection 



-37-* 



JP-A NO. 2000-175019 



23. Rule selection 

24 . Change location decision 

25. Change 

31. Moving image 

Rule aggregation 
Use rule 

Embedding information 
Information embedding moving image 
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FIG. 3 
T$tfe Start 

221. Fetch a back frame positioned K frames behind a 
notice frame 

222. Split the noticed frame into plural blocks 



JP-A NO. 2000-175019 



223. Notice the first block 

224. Calculate motion vector between the noticed block 
and the back frame 

226. Notice the next block 

225. All blocks processed? 
#17 End 
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Fig . 4 

220a. Block 

220b. Motion vector 



Fig. 5 

1. Rl: If the magnitude of motion vector is less than 
Al, select watermark intensity specification table 1. 

2. R2: If the magnitude of motion vector is equal to or 
greater than Al and less than A2, select watermark 
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intensity specification table 2. 

3. R3 : If the magnitude of motion vector is equal to or 
greater than A2 and less than A3, select watermark 
intensity specification table 3. 
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Fig. 6 

320a. Block change ease level 

320b. Block change rate (number of change pixels/number 
of pixels within block) 

0 £A -t 0 . 5 7^$! : Equal to or greater than 0 and less than 
0.5 

: Equal to or greater than 0.5 and less 

than 1 . 0 
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Fig. 7 

Start 

241. Determine block change ease level 

242. Determine the change ease rank of pixel within 
block 

243. Notice first block 
Decide change rate of noticed block 
Select change pixel 
Notice next block 
All blocks processed? 
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$£T End 
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Fig. 8 

320a. Block change ease level 

320b. Block change rate (number of change pixels/number 

of pixels within block) 

320c. Change amount of each pixel 

0 J^Jt 0 . 5 TfcYiSj : Equal to or greater than 0 and less than 
0.5 

0 . 5 jEA_L 1 . 0 : Equal to or greater than 0.5 and less 

than 1.0 
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Fig . 9 

21. Input-output 
20. Control 

22. Inter-static-image motion detection 

26. Change rate 1 decision 

27. Static image analysis 

28. Change rate decision 

24. Change location decision 

25. Change 

31. Moving image 
36A. Change rate 1 
33 . Use rule 
36B. Change rate 2 
37. Change rate 
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34. Embedding information 

35. Information embedding moving image 

[H i 01 




Fig. 10 

2 1 . Input - output 
20. Control 

27 . Static image analysis 

22. Inter-static-image motion detection 
29. Change rate adjustment 

24. Change location decision 

25. Change 

3 1 . Moving image 
33 . Use rule 
36B. Change rate 2 
37. Change rate 
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34. Embedding information 

35. Information embedding moving image 
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45 ®<7>z7u vf&WO StSl fc^-C#5o 1O07U- 

A3ri?f/$-r£j-fe2 7 o ofls^T'p £Nflaos:3==i 

T, 2 7 0 O/Nfl^ny^ilBe 
<9^T£^ 1 ify 4 5 O/NfllO^n y $ 

50 9 St5:t#-e*5„ 
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[0 0 2 8] h^*-^* 

^T, ±E*JE*»fftJfcA'— *^2 4*»*i-5««tt 
■"Cl«**»S*««tW*< U K> hft$& "0" 3rS 
* r tr»*Ht, ±B»*A— ^ 2 4 5Pf*ffi 

u - A ft "CSV > left » UfcffiKBB« fc fc 5 «*<*>Bfc:/ 
[0 0 2 9] ±aibfcp^^n 3,^ \f y hm^tCDM 

*r iot^D y * SOt§ &*W«*«0r"C# 5 ± 5 
trLT*5< 0 

[0 0 3 0] ft, S*»LflMBtt, ±IS6 tfy MMifcft 

*>\z&\£yYkWto7w9k SrSf JK# # 5 ft to 19 ir , 
^Jx.tf, ±156 try Hf«-e*Et»#S6 4«Scd#^: 

Ktf>W«fr J:oT*HSi-SIBIfeyn ^<0JffiMfcft«:Sd 

fi, 0-12 7©12 8SHf*>S^ tlW^LT 

M-fSfttotUr. #®*<z>»*^M«lSfgir@^L 

[0 0 3 1] B3tt, S-ihHP^ib^^ttS/^-^V 2 2 O 
fftt^o-? 1 *- hi**-*-. rr^RitSifftlll 

^LBE^OSSIj , pp. 44-47, 1 9 9 6^^tl5^JO 

fc-t stable ftt&ttjSSSrftt^ 
-eft***, *ftir o^xti*BS LfeKW i i - 

[0 0 3 2] Btr^-f J: 5ir, ftl7 7^^3 l 
S 7 J* i "9 K:7 Afctf^tr#g Lfdi 7 
^A^#i7 I — At IT® Offll (^fy^22 
1) , f@7U-A^l 6I|X8li^/o 
#J1"S (^7^2 2 2) o ±H5*S ^ l/-AOta^ 



h^Sr«H«^l — ?^2 3^atf (77^2 2 
4) o jfclr, *B7i^-A(0^uy^Srfti3iLfcd*5 
05 fr&nfeL (^7^2 2 5) U ^n^/^W 

6) , m^^t h/^ttl^75'/2 2 4Sri^»9ig-fo 
[0 0 3 3] ±E»Jh!Bm]»St*a^**, H4li:*t 
10 £?(r, ||7l/- A^/n^^ 2 2 0 a t*tfSUT 
tb^* h/W2 2 0 hSr*-***^ h/u^-^2 2 

iRA— 2 3 tr-^x. r k KttZo 

[0034] mmm^—^^ 2 3 0Hx.«, m 5 ir 

15 /TtJ:5C ttt^ h^o***(ri^i:TaW-r-<# 
1, R2, R3. . . «T*TU ^S^l/-^rtcD#yn 

y?m^ iEBttfttti^-^^2 2-c*tusnfc«i# 
^ h^a t#iSLfcaHijsr±Eaai**7 

[0035] mmfeyy-i^z 2izmM£ti%mm 

m&mfeT-zf/v) 3 2 0^ IU6tr^-rJ: 
?(c. »-ih®irfcits^n y^<DjE3E*A«3 2 o a £ 
25 T'n jy^rtJH*3 2 0 b tOHMJliUTl^o _h 
EfMM3 2 0 a(^ ^oy^gC^gS^t 

fit, -t"fcto*>, l W^S ^Sua!^ 1 2 8^PSo5 ^cofpj 

D^^^If 3 20bl^ 1 y^rt(DfSl%(DHSl 

mmmt ITlK^^o Sot, ±IE^M*«, 3^35 
B^co?ig Wn;y ^ T*oii*^3CJESa t ^^IB#tr 

[0 0 3 6] W^SflSR 1, R2, R 3 . . . tt, B# 

^ h/i/fcjftli] (aft^x-^3 2 0) t<DBB«*E 

[0 0 3 7] i7it K3E«Bf*S^— ^> 2 4 Oplffl 
45 7n~ft- h£^ 0 S@7U-A^P5/ 

(77^^24 1) 0 *tl7^l:oPT, «i 
tf, «^«f«iiflt^|»Xtt, D-2, IJ79-D- 
18^ mi347~1353H{rlS«^ftfcfttt^tO 
50 IS7^^yy/(aori, AW^B trfiftift-cg: 
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-YXBfc* (iB*tt&£) fcys&ifcu 

^M«i:7E^lil«t*rJI«ri:5-r ttrioT, ^Mfi^r 

HHfe:? U'-AS:* g 7 u — A t mm\cmm<v7* y ? fc 

86 fcjj*bfc^3ES«ffi 3 2 0 a £#:#>5 0 
[0 0 3 8] _b!EJg:7U— AO#;/ny^K::|3^ 

2 C»LTlR|-(7)jWSttf I^Di/ci^, fcU p 
l p 2 «fc D t>8W#»«#*s*tt;h,ii, p l <d1j 
#p 2 J: Ot>«tt*lE*B Scfc*ir\ 
[0 0 3 9] ro^ffiT% f@7l/-^*jS0^ny 

0) SrBt^fflU ±lS«IlJ(c^or, ±85^X5/^2 4 
lT»IttiUfe5t»yD^^IIM3 2 

0 a ^£t5^o 7^«I$3 2 0 b ?r*fe5 
ry/2 4 4) o rtUCJzoT, ±|S*B^a5/^rto 

1 2 8fiOt°^i?/WQ5 

*^>>:/2 4 2-e^i6fc^M^*fW^iisv^t 

O^feWS^, JLfe^n 5/^^^Jg*3 2 0 bX^£ft± 
«Ot°^ir/W£^KU ^JgA— ^>2 5(C}g^i-5 
(^f y/2 4 5) 0 «|7U-A^ny 
^^ov^T&aiSr^^^SrW^L (xr^2 4 
6) % i7u ^^*W^ v i^(:|j::^wf 

^T7^2 4 4(CI5 0 

[0 0 4 0] ^^2 5TU ^Alt^r^A-. 



10 [004 2] B8li, B6t* IstcMm 3 2 0 <D$EMM 
Sr*-t\ rofiaj-Cfi, ^Dy^^!S$M3 2 0 at 
*fj£tt, ^n;y^rt^H$3 2 0 bOfe 
te<£>3W*^M;B: 3 2 0 c ^fiLt^< :^:j:ot, 

[0 0 4 3] 0 911. 
5i*^«2 0l|j«W^i-««B^oy^|2-e*>5 0 ««g 
20 5/^ 2 0-2 8(l4 , *:^&S2^||^Ti"5^/l- 

— ^ N «Sfg^n 5/^31-3 7I11E18 

i£fi3F^c^/$;£*i5^— ^^r^/^S:^LT*5t), IH 
2 -CRM Lfc Jg 1 %tt«ftj i *ai-«K*|ctt|R|-0«F* 

25 K«&0«fliJ«:flljgcU f|7l/-AO#7n^fi 

ffiffl«aiJ^T-f^3 3(C*&J^bT*5V^c 2 

30 [0 044] ^Jg* 1 O&r^A— 2 6 »JtHIW 
^SbttfcttJ/W-^ 2 2 ^bS«tKo/c*@ 7 Aft 

3E*l 77>f/V3 6 Atc|5tg"r^) 0 ±fE^oy^rt*H 
35 *SrftSf4, Kj#-<^ h;^t<^^«otfI$ 

QBB«§r£Sl Lfc**^— ^S:#flBi-5 n t kc X o 

[0 0 4 5] ^±m^^—^> 2 7 Ht, 12 -CRM L 
40 fc^55ffl^fifi^*^i— 2 4 J:o 
T^p^Ofim3 2 0 aSrJIUU ffifflSIO 
77^3 3*Cffl«Six*:aaij3 2 OMot, S@y 
O7^rt^M$3 2 0 bS:ftSU ^(D^^^m 
277^/1^3 6 Bldtett-TSo 
45 [0 0 4 6] ^J:^;:, pj— co#@y 

tc^CT*^Ly L c|ES^2(^5 *Sr, «3E*ft5£ 

^-^^2 8(cj:oril^i-5 0 $I$ft^-fy2 
50 8(1, Jl»«*3E*OB[KS:«jfei-5fc*^, ± 
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W7 7 ^^3 7 {rlEtt-T-So fcU ffi*ii3M£<z>&a» 
Uf*W»te&j£$fcLfcV'>it£-|4, ±SE2otf>^MJ$<£> 

[oo47] ^mmm^-f-^ 24'ii, a? i mfe 

[0 0 4 8] ±5£LfcJ:5Ks f2C»Jli, ftWft 

[0 0 4 ?] bioii, Xftwicxzw.^m^mmm 

mzfo 2 0~2 9««f*:*!l.S^e2^||=fT-r 
iV-^s (^niT^A) , «te^ny^ 3 1~3 7«tSB 
tfeSIS 3 ftlCMffcZtlZ? 1 —? 7 T-Y/l'Sr^ LT*3 19 , 
12, H9t?»WbfcJ(Sl, ^2Hlfi«ilJt#ii-rs^ 

[0050] §±iM^-f^2 7 is, m2 mnmt 
tt-rs. ^je*88^/v— f^2 9«, m±mmm^^m 

MiftSLt, ±IE^ 1$ 2 7 7 ^ 3 6 B i^ffi 
7^3 7»c|E*-r<5. ^S«^-fy2 4' 

[0 0 5 1] ±|Efg 3 »*lfe^Jt?li, »ih®6tH±K(C.t 
[0 0 5 2] gUKDjg 1 ~3? 3 C0H»|T'(4, Bb®^*t 

<Z5*D<. =.yfy'7^ix5M5 2oro|4S^«L 



* £ *V5 HftJSW Wffe<D 3 >T > y tc fcJSffl aim Xh 
*aa$IH2(c:a;?)y7' h!7aiT (;i/-f>2 0~2 9) 

05 T&ii-Dismo-mt&mik&titi^- w^rxmm. 

L,Xi>£\,\ 
[0 0 5 3] 

io tt«©ae3E«Bf, *3io t /*fcf4^Ms*^*ig^-rs 

ri:(wioT, 3>x>-y«i (If, 05* 

[0 1 ] *»w©«^»a»LW«»*a*&**-rsfc 

20 *©a-K^i7M„ 

[03] a2fc$3tt5#JkWrott#ttffl^— fV2 2© 
i¥»^t7n-f + - Ho 
25 [04] ^JhSWtbt^m/V— ^>-2 2|Cj;57'o5'^ 

[05] B2fc:fc»*5«HlJ»^u— ^>-2 3##*.5jfc 
giJ)lW©*lJ^«^^*-t-0 o 
[0 6] fi?«ffiji^aiU© 1 0iJ£r*-r0o 
30 [0 7] 0 2(C*3(t5*M©^^-^V2 4©f¥*ffl 
^t7d-f+- K 

[0 8 j mm®.&%.mM)i<7>m<z>m&7frrm. 

[0 9] *^M(cJ;5lg^3gA>L-ffifffiffiJi)ii^©^2© 

35 [01 o] ^m\cx^mf-m^unmmi!b^^.(o^3 

[ft*a>BMI§] 

i : xmt/mm.. 2 : * $Mmmw., 3 : effiti. 2 

0 : ffilJ«/U-^V, 2 1 : Atti/J^-^-V, 2 2 : »± 

40 iSKKj^^m/i'-^v, 2 3 : mmft.fr-f->. 2 

4 : £JEffi@frftJ£>'l'— f 1 ^, 2 5: ^H/U— 2 

6 : 1 ro^/t/U— g^v, 2 7 : #ih®^/U— 9^ 
^ 28 : ^M^^/U— ^-V, 2 9 : £JE*WfiA'— 
^y, 3 1 : 1I7 7^/K 3 2 : r 

45 3 3 : ffiffl^fllj7r-f/K 3 4 : #Atf $6^ 7 ^, 3 

5 : ftitm AW}W7r4fr, 36 :fI$77-f/K 3 

7 : Iiagl^T'f/W, 2 2 0: K#-<^ h^SEffi^ 

3 2 0: «5**jeaau 
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(72)^bj# mm ?h 

<72)$8W# #*t 

#^JIim«fftr|TF^I2:S"fflBT292#it& t* 

W^JH*«IST|TF«IE:*fflBT292#iffi t* 



(72) mm^ YnH mx 

W^JH»«^rtTF«E^fflBT292#lt ft 



30 d2) *r\u mm 

*ina^&Tiig±i6Tiif it /•? 

(72) fpSi 

*3EJR»*ftifj*:fcJB,6TB 1#1 * /< 
35 -Y T *a^tfc»^W^fllf ft 

(72)^BJ# «c«F * 

JPRf&fnJII&lfciBiJII 6TI 7#35-§- 

F^— 5B057 AA20 CA12 CA16 CB12 CB16 
40 CB19 CC02 CE08 CG07 DA08 

DA17 DC02 DC05 DC36 
5C063 AC01 BA12 CA29 CA36 CA40 

DA20 EB39 EB46 
5C076 AA14 AA36 AA40 BA06 
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